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Abstract 

Background: At present, there are no guidelines on prevention and management of postpartum haemorrhage in 
primary midwifery care in the Netherlands. The first step towards implementing guidelines is the development of a 
set of quality indicators for prevention and management of postpartum haemorrhage for primary midwifery 
supervised (home) birth in the Netherlands. 

Methods: A RAND modified Delphi procedure was applied. This method consists of five steps: (1) composing an 
expert panel (2) literature research and collection of possible quality indicators, (3) digital questionnaire, (4) 
consensus meeting and (5) critical evaluation. A multidisciplinary expert panel consisting of five midwives, seven 
obstetricians and an ambulance paramedic was assembled after applying pre-specified criteria concerning expertise 
in various domains relating to primary midwifery care, secondary obstetric care, emergency transportation, maternal 
morbidity or mortality audit, quality indicator development or clinical guidelines development and representatives 
of professional organisations. 

Results: After literature review, 79 recommendations were selected for assessment by the expert panel. After a 
digital questionnaire to the expert panel seven indicators were added, resulting in 86 possible indicators. After 
excluding 41 indicators that panel members unanimously found invalid, 45 possible indicators were assessed at the 
consensus meeting. During critical evaluation 18 potential indicators were found to be overlapping and two were 
discarded due to lack of measurability. 

Conclusions: A set of 25 quality indicators was considered valid for testing in practice. 
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Background 

Postpartum haemorrhage (PPH), internationally defined 
as >500 mL of blood loss within 24 hours after child- 
birth, remains one of the leading causes of severe mater- 
nal morbidity and mortality worldwide, especially in low 
resource countries [1]. 

The definition of PPH, however, is not unified; in 
high-resource countries PPH is often defined as blood 
loss of at least 1000 mL, while a woman in good health 
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can tolerate up to one litre of blood loss without show- 
ing early signs of shock [2-6]. Over the last 15 years, an 
increase in PPH has been observed in high-resource 
countries. The reasons for this remain unclear [7]. 

Almost one third of Dutch women (32.7%) give birth in 
'primary care' which is low risk care supervised by a mid- 
wife (99% of births) or general practitioner (1% of births). 
Of all births in primary care, 64% occur at home [8]. 

In the Netherlands the overall prevalence of PPH (de- 
fined as >1000 mL blood loss), is 5.9%. Of all births in 
primary care, 3.4% is complicated with PPH [8]. When 
PPH occurs, women are referred to secondary care and 
treated by obstetricians. In a home birth setting, women 
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are then transferred to hospital by ambulance. Audit of 
care provided in case of severe complications in preg- 
nancy and childbirth has shown considerable room for 
improvement of PPH management [9,10]. 

The Dutch Society of Obstetrics and Gynaecology 
(NVOG) published guidelines concerning prevention 
and management of PPH for women giving birth in hos- 
pital supervised by an obstetrician [4]. At present, how- 
ever, there are no guidelines on prevention and 
management of PPH in primary midwifery care in the 
Netherlands. Although published obstetrical guidelines 
can be and are used in primary midwifery care, the 
unique conditions in midwifery care (such as low-risk 
profile and birth at home) call for guidelines specifically 
designed for primary midwifery care. 

The first step towards such guidelines is determining 
applicable items, preferably by using quality indicators. 
Quality indicators are derived from outcomes of studies, 
historical data and expert opinions and are defined as 
measurable elements of practice performance for which 
evidence or consensus exists. They can be used to assess 
and improve quality of care provided to the woman [11]. 
The aim of this study is to develop a set of quality indi- 
cators designed for the prevention and management of 
PPH in primary midwifery care. 

Methods 

The RAND modified Delphi method was used to develop 
a set of quality indicators for prevention and management 
of PPH in primary midwifery care. This method has been 
proved valuable as a systematic method using current sci- 
entific evidence in conjunction with expert opinion 
[12-15]. For this study, ethical approval was not required. 

Indicator development procedure 

The procedure for quality indicator development con- 
sists of five steps: (1) composing an expert panel, (2) lit- 
erature research and collection of recommendations, (3) 
questionnaire, (4) consensus meeting and (5), critical 
evaluation (Figure 1). 

Step 1: composing an expert panel 

In order to capture all aspects of care concerning prevention 
and management of PPH, members were selected with ex- 
pert knowledge in (at least) one of the following domains: 
primary midwifery care, secondary obstetric care, emergency 
transportation, maternal morbidity or mortality audit, qual- 
ity indicator development or clinical guidelines development 
and representatives of professional organizations (Royal 
Dutch College of Midwives [KNOV] or NVOG). 

Step 2: literature research and collection of possible indicators 

In order to identify possible indicators for PPH, first 
PubMed was searched using the following keywords: 



'postpartum haemorrhage) 'home birth) 'low-risk birth, 
'prevention' and 'third stage of labour' in combination 
with 'guideline' or 'quality indicator'. The Internet was 
searched for reports and statements on PPH, especially 
in primary (midwifery) care. Following this, international 
guidelines, protocols and consensus statements were re- 
trieved and collected. Indicators used in secondary ob- 
stetric care concerning prevention and management of 
PPH were included. Finally, in order to complete the 
preliminary set of possible indicators, manuals of obstet- 
ric emergency courses regarding prevention and man- 
agement of PPH were studied. Due to the lack of a 
unified definition of PPH, PPH defined as 500 mL and 
1000 mL were categorized separately. Some items, such 
as surgical procedures and embolisation, were clearly 
not applicable in primary care, and therefore deemed 
not relevant for this study. 

Other items needed rephrasing for clarification of the 
possible indicator. The list of possible indicators was 
categorised into five domains: prevention, >500 mL 
blood loss <1000 mL, >1000 mL blood loss <2000 mL, 
>2000 mL blood loss and organization of care. 

Step 3: questionnaire 

A questionnaire listing all possible indicators was sent to 
all panel members via e-mail. To facilitate decision- 
making, the source(s) and relevant literature citations 
for each potential indicator were provided. 

Panel members were asked to score the possible indi- 
cators on a nine-point Likert scale, ranging from 'one', 
being a poor quality measure of care, to 'nine] being an 
excellent quality measure. In addition, panel members 
had the option of selecting 'not assessable'. The respon- 
dents were asked to score each possible indicator with 
respect to their impact on both 'health gain' and 'overall 
health efficacy'. Health gain was defined as: 'An increase 
in the health of individuals or population' and overall 
health efficacy was defined as: 'prevention of unneces- 
sary medical treatment and promotion of cost- 
effectiveness' [14]. In addition, panel members were 
given the opportunity to provide comments or suggest 
additional indicators. 

All data were collected and analysed using the Statis- 
tical Package for the Social Sciences (SPSS), version 17 
(SPSS Inc., Chicago, IL, USA). The median panel rating 
and the amount of dispersion of ratings between panel 
members were calculated for each potential indicator. 
The comments and newly proposed quality indicators 
were collected. For an optimal assessment of all possible 
indicators offered to the panel, no indicators were 
discarded between this questionnaire round and the 
consensus meeting. All newly proposed possible indica- 
tors were added to the list. Overall agreement on each 
item was defined as 75% or more of ratings within a 
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Composition of an expert panel 
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Literature search, selection of possible quality indicators 
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79 possible indicators 
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Transcription into 25 quality indicators 



Figure 1 The process of quality indicator development according to the RAND-modified Delphi method for prevention and 
management of PPH in primary care in the Netherlands. 



panel being in the lowest (1, 2, 3,) or the highest tertile 
(7, 8, and 9). The subsequent consensus meeting focused 
on indicators with low agreement. 

Step 4: consensus meeting 

The expert panel was invited to a face-to-face consensus 
meeting. At the onset of the meeting, each panel member 
received the list of possible indicators, together with their 
own ratings from the questionnaire. The median rating 
and the frequency of responses for each possible indicator 
were also provided. Finally, panel members received the 
list of newly introduced potential indicators from step 
three. Individual ratings of the other panel members were 
kept confidential. Subsequently, the panel was divided into 
three groups of either four or five participants, every 
group consisting of at least one midwife and one 



obstetrician. Each group was assigned one or two domains 
(as described in step two), and were asked to evaluate the 
practical applicability of each possible indicator. 

Each group (moderated by one of the authors) focused 
on indicators not unanimously agreed upon in the first 
questionnaire round. Indicators where the range of dis- 
agreement was widely spread were also discussed. The 
aim was to assess if there was genuine clinical disagree- 
ment about the validity of possible indicators or if there 
was a problem with phrasing. After the three groups 
assessed their assigned domains, the entire panel 
discussed potential indicators that were not agreed upon. 
After the panel meeting, the members were asked to rate 
all the indicators again. The final ratings were analysed 
in a similar manner as in step three. Analyses were 
performed based on the RAND/UCLA (University of 
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California Los Angeles) appropriateness method [15]. 
An indicator was considered as 'valid' if there was an 
overall panel median score of eight or higher and if 
'agreement' was reached between panel members. 

Step 5: critical evaluation 

In adherence to the RAND method, the core panel crit- 
ically evaluated the indicators with high agreement in 
step four. Emphasis was put on applicability, feasibility 
and measurability. Some indicators were modified or 
combined due to overlap between categories or prag- 
matic reasons concerning implementation, resulting in a 
final consensus-based set of indicators. 

Each indicator was assessed and rephrased to define a 
numerator and denominator: the number of women in 
whom a specific test or intervention has been performed, 
divided by the number of women in whom this test or 
intervention should have been performed. By this last step 
use of the indicator can establish the percentage of adher- 
ence when evaluating quality of care. 

Results 

The process of development of the indicators can be 
seen in Figure 1. 

Step 1 

After selecting experts in one of the previously described 
domains, a panel of thirteen members was assembled 
consisting of five midwives (one of whom is first author), 
seven obstetricians (including three of the authors) and an 
ambulance paramedic. All midwives and obstetricians work 
in maternity care and are actively involved in at least one of 
the domains as described in the Methods section (Step 1). 

Step 2 

A literature search resulted in a list of publications 
from which possible indicators could be extracted 
[2-4,9,10,16-30]. From these publications, all possible 
quality indicators for women at increased risk of PPH in 
secondary care were collected. More than half of the indi- 
cators were immediately discarded, as they are not applic- 
able in primary midwifery care (e.g. surgical procedures 
and embolisation). Two studies on PPH and homebirth in 
an industrialised country were found. The authors made 
recommendations on referral in case of PPH and/or 
retained placenta after home birth [9,10]. These recom- 
mendations were incorporated in the list of possible indi- 
cators. This survey resulted in a list of 79 possible 
indicators, which were categorized into five domains as 
described in the 'participants and methods' section. 

Step 3 

A questionnaire composed of the 79 possible indicators 
was sent to all panel members via email. The ambulance 



paramedic only rated possible indicators within his field 
of expertise and rated some indicators 'not assessable'. 
The expert panel proposed seven additional possible in- 
dicators. Finally, a list of 86 possible indicators was pre- 
pared for assessment at the consensus meeting. 

Step 4 

All panel members attended the meeting. After discussing 
and reassessing the 86 possible indicators, 45 recommen- 
dations were rated 'valid': four on prevention, nine on 
500-1000 mL blood loss, 12 on >1000 mL blood loss, 14 
on >2000 mL blood loss, and six on organization. The 
remaining 41 indicators were rated 'not valid' and subse- 
quently excluded (Figure 1). 

Step 5 

During critical evaluation by the core panel 18 potential 
quality indicators were found to be overlapping and two 
were discarded due to lack of measurability. Finally, a set of 
25 potential indicators were transcribed into 25 quality in- 
dicators for prevention and management of PPH in primary 
midwifery care in the Netherlands (Table 1). The indicators 
each now contain a numerator and denominator, i.e. in case 
of PPH; the number of women with PPH who had an intra- 
venous line is divided by the number of women with PPH. 

Discussion 

A RAND modified Delphi method approach was used to 
develop a set of 25 quality indicators. This is the first set 
of quality indicators concerning prevention and manage- 
ment of PPH in primary midwifery care in the 
Netherlands, to be used to assess care in case of PPH in 
primary care. This is an essential contribution to the de- 
velopment of guidelines of PPH in midwifery care. 

The use of uterotonics, placing an intravenous line and 
quick referral in all cases of PPH were considered of great 
importance by the majority of the panel and thus incorpo- 
rated in the final set. Possible indicators of the management 
in case of PPH > 2000 mL were either accepted or rejected 
with minimal dispersion. For some indicators however, as- 
sessment of validity was a source of discussion. For ex- 
ample, the routine use of oxytocin was hotly debated. As 
shown in a nationwide survey, most obstetricians consider 
this as part of standard care. In midwifery, though the use 
of uterotonics has increased over the last decade, this is no 
standard practice [30]. Currendy, the Royal Dutch College 
of Midwives has not issued a guideline for women at low- 
risk of PPH or made any statement concerning manage- 
ment of third stage of labour. Also, in Dutch midwifery 
schools, no unambiguous policy is taught on the routine 
use of uterotonics. In the process of guideline development 
and implementation, routine use of uterotonics might be 
an item for further discussion, especially also because of the 
high prevalence of PPH in our country. 
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Table 1 Final set of quality indicators for the measurement of PPH-care in primary care 



Median (n) Agreement (% of panellists 
with score of 7, 8 or 9) 



For prevention of PPH, the midwife should; 

1 Antenatally: identify elevated- or high risk of PPH and agree on preventive strategies* 1 

2 At birth: identify elevated- or high risk of PPH and agree (or adjust) preventive strategies* 1 

3 If high risk of PPH is assessed: have birth occur in hospital supervised by the obstetrician. f 

4 Routinely administer uterotonics (at least 5 IU oxytocin intramuscular). + 

In case of blood loss >500 mL, without signs of shock the midwife should; 

5 Measuring blood loss by weighing. + 

6 Homebirth: in case of retained placenta; refer to secondary care after 30 minutes 



Midwifery supervised hospital birth: in case of retained placenta; refer to secondary care after 
30 minutes 



8.5 (12) 
8(12) 
8.5 (12) 
9(12) 

9(12) 
9(13) 
9(13) 

9(12) 
9(12) 
9(13) 

7 (12) 



1 00 
100 
100 
83,3 



92,3 
75 

83,4 
83,3 
100 

/5 



8 Homebirth: if blood loss is not ceasing, refer to secondary care. + 

9 Midwifery supervised hospital birth: if blood loss is not ceasing, refer to secondary care. + 

10 Treat PPH as uterine atony (and apply bladder catheterization, uterine massage and oxytocin) 
until proven otherwise. 

1 1 Post placental: if blood loss is not ceasing despite administration of uterotonics, examine 
for vaginal and perineal lesions. 1 

In case of PPH of >1000 mL and/or signs of shock, the midwife should; 

12 Inform the secondary caregiver (obstetrician). 9 (13) 100 

13 Start an intravenous line and supply with fluids, using 0, 9% sodium chloride. 8 (13) 100 

14 Monitor vital signs frequently (pulse, blood pressure, respiratory frequency). 8 (13) 92,4 

15 Regardless of oxygen saturation, provide patient with 10-15 litre oxygen via non-rebreathing 9 (13) 84,6 
mask. 

In case of PPH of > 1000 mL with signs of shock and/or >2000 mL blood loss the midwife 
should; 

16 In case of persisting haemorrhage with signs of shock, perform uterine and/ or aortal compression. + 8 (12) 83,3 

17 Secure a second intravenous line (14 gauge). 9 (13) 79,9 

18 If the patient has reduced consciousness due to hypovolemic shock, call for (paramedic) 9(13) 83,4 
assistance in order to establish an open airway. 

1 9 Immediately transfer patient to secondary care. 1 

Concerning cooperation and training; 

20 Within every regional obstetric collaboration' a regional PPH protocol should be present, based 9 (13) 91,7 
on national guidelines. 

21 A regional PPH protocol should be the basis of regular audits. 9(13) 83,3 

22 The midwife is aware that ambulance transportation in case of PPH or retained placenta is always 9(13) 91,7 
of the highest urgency category. 

23 After each PPH with >2000 mL blood loss, the multidisciplinary team should debrief the situation. 8 (13) 83,4 

24 Within the regional obstetric collaboration' an annual training in obstetric emergencies should 9 (13) 100 
be provided. 

25 In a homebirth situation, anticipation on possible ambulance transport is necessary; make sure the 9 (13) 100 
patient is at an accessible place for (all) caregivers in time. 

* Preventative strategies imply consultation with an obstetrician to determine policy regarding PPH prevention e.g. birth supervised by obstetrician, or birth 
supervised by midwife, but in hospital with intravenous access prior to birth. 

£ Regional obstetric collaboration; a quarterly meeting with obstetricians and midwifery practices within a region in the Netherlands where policy, collaboration 
and practical agreements are discussed. 

f The ambulance paramedic did not rate these items; it was not within his field of expertise and stated these as 'not assessable'. 



(12) Added in 100 
second round 



Although the effectiveness of comparable indicator de- literature search, possible indicators may have been 
velopment initiatives has been proven, there are limita- overlooked. However, the expert panel was given ample 
tions to this method [13,31,32]. Despite a thorough opportunity to propose additional items, both in the 
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questionnaire round and during the consensus meeting. 
Of the seven additionally proposed indicators, three 
were incorporated in the final set [32]. It is well- 
documented that panel composition influences the out- 
come of the indicator-development process [33]. If more 
than one discipline of health care providers is included 
in an expert panel, lower agreement in rating between 
members are found, compared to when only expert in 
one discipline make up the panel. In this study, the panel 
consisted of a heterogenic group of professionals. There- 
fore, in case of high agreement, that indicator can be con- 
sidered highly valid. Furthermore, it has been shown that 
the applied method (using a higher cut-off point for deter- 
mining consensus with an overall median rate of 8 out of 9) 
enhances the reproducibility of ratings if a different set of 
panellists would rate the indicators [13]. 

In our literature search, many studies on PPH and 
homebirth originated in low-resource countries [19,24,34]. 
However, home birth in these countries is rarely a well- 
considered choice by women, and frequently being the re- 
sult of poverty and lack of accessibility of health facilities. 
Therefore, it was often impossible to extrapolate recom- 
mendations into a western primary care setting. Only a 
few studies contain relevant information on home birth 
and referral in industrialised countries [9,35]. Thus, the 
scientific evidence base was limited in this area of primary 
care and necessitated the use of expert opinion in addition 
to available evidence. Due to this finding, we conclude that 
referral in case of PPH at home to hospital in indus- 
trialised countries is under-researched. 

All quality indicators need to be validated, in order to 
ensure the clinical relevance [13,31]. Currently work is 
underway to validate this set by assessing collected cases 
of PPH in primary midwifery care in the Netherlands. 

This set of indicators provides us with an instrument to 
assess the care commencing in a primary midwifery set- 
ting, before being transferred by ambulance to hospital. 

Conclusion 

A set of 25 quality indicators for prevention and manage- 
ment of PPH in primary midwifery care in the Netherlands 
was developed. This is the first set of quality indicators which 
may serve as an assessment tool for prevention and manage- 
ment of PPH in primary care. This is of great interest, as the 
incidence of PPH is rising worldwide. Furthermore, existing 
guidelines for secondary care can be combined with these 
findings, so care throughout the care chain, including ambu- 
lance referral, can be thoroughly evaluated. 
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